18.05 Lecture 11 
February 28, 2005 



A pair (X, Y) of random variables: 

f(x, y) joint p.f. (discrete), joint p.d.f. (continuous) 

Marginal Distributions: f(x) = J2 y f( x ^y) ' P- 1 - °f X (discrete) 

f(x) — J f{x 1 y)dy - p.d.f. of X (continuous) 

Conditional Distributions 



Discrete Case: 



P(X = x\Y = y) = 



F(X = x,Y = y) 
p(y = y) 



P = = f{x\y) conditional p.f. of X given Y = y. Note: denned when f(y) is positive. 

f{y\x) = conditional p.f. of Y given X = x. Note: denned when f(x) is positive. 

If the marginal probabilities are zero, conditional probability is undefined. 



Continuous Case: 

Formulas are the same, but can't treat like exact possibilities at fixed points. 
Consider instead in terms of probability density: 
Conditional c.d.f. of X given Y=y; 



¥(X <x\Ye [y-e,y + e]) = 
Joint p.d.f. f(x,y),F(A) = J A f{x,y)dxdy 



V(X<x,Y€ [y-e,y + e]) 
P(YG [y-e,y + e]) 



y + s 



y-s 



IyTj'°° ocl f^,y)dxdyx ± 

Ase->0: 

I- OQ f(x,y)dx _ f(x, y)dx 
C 00 f(x,y)dx~ f(y) 

Conditional c.d.f: 
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f x ' fix, y)dx 

nx<x\Y = y) = J -™y {y ' ) y> 

Conditional p.d.f: 

Same result as discrete. 

Also, f(x\y) only denned when f(y) > 0. 

Multiplication Rule 



f(x,y) = f(x\y).f(y) 

Bayes's Theorem: 



_ f(x,y) _ f(x\y)f(y) _ f(x\y)f(y) 



f(x) J f(x,y)dy J f{x\y)f(y)dy 

Bayes's formula for Random Variables. For each y you know the distribution of x. 
Note: When considering the discrete case, J — > 

In statistics, after observing data, figure out the parameter using Bayes's Formula. 

Example: Draw X uniformly on [0, 1], Draw Y uniformly on [X, 1] 
p.d.f.: 

f(x) = 1 x 7(0 < x < 1), f(y\x) = -L_ x I(x < y < 1) 

1 — x 

Joint p.d.f: 

f(x,y) = f(y\x)f(x) = ^— x 1(0 < x < y < 1) 

Marginal: 

f(y) = J f{x,y)dx = j V Y ^dx = -\n{l-x)\l = -\n{l-y) 

Keep in mind, this condition is everywhere: given, y S [0, 1] and f(y) = if y ^ [0, 1] 
Conditional (of X given Y) : 



Multivariate Distributions 

Consider n random variables: X\, X2, X n 

Joint p.f.: f(xi,x 2 , —,x n ) = P(Yi = x\, ...,X n = x n ) > 0,J2f = 1 
Joint p.d.f.: f{x\, X2, x n ) > 0, J fdxidx2---dx n = 1 
Marginal, Conditional in the same way: 
Define notation as vectors to simplify: 

X = (Xi,^.,X n ),~X = (X!,...,X n ) 

X = (Y , Z) subsets of coordinates: Y = (X\, X/.), ~y = {y\...yk) 

Z = (Xk+i, ...,X n ),~z — {z\...z n -k) 

Joint p.d.f. or joint p.f. of X , f(~x) = f(~y, ~~z) 



33 



Marginal: 

f(y) = J f f (?) = J na^w 

Conditional: 



fit) f ftf\it)f{-Z)jnt 



Functions of Random Variables 

Consider random variable X and a function r: M — > R, 

y = r(X), and you want to calculate the distribution of Y. 

Discrete Case: 
Discrete p.f: 

f(y) = ¥(Y = y) = P(r(X) = y) = ¥(x : r(x) = y) = ]T F(X = x) = ]T f(x) 

x:r(x)—y x:r(x)—y 

(very similar to "change of variable" ) 

Continuous Case: 

Find the c.d.f. of Y = r(X) first. 

P(Y <y) = F(r(Y) <y) = ¥(x : r(x) <y) = ¥(A(y)) = [ f{x)dx 

JA(y) 

p.d.f. f{y) = -§- y j A{y) f{x)dx 
** End of Lecture 11 
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